We study characteristics of receptive fields of units in deep convolutional networks. The receptive field size is a crucial issue in many visual tasks, as the output must respond to large enough areas in the image to capture information about large objects. We introduce the notion of an effective receptive field size, and show that it both has a Gaussian distribution and only occupies a fraction of the full theoretical receptive field size. We analyze the effective receptive field in several architecture designs, and the effect of sub-sampling, skip connections, dropout and nonlinear activations on it. This leads to suggestions for ways to address its tendency to be too small.

This allows us to interpret the latent variables as object positions and lets us train the network without supervision.

For an example unit j, if it shows activity for input images having white pixels at 6 dva to 8 dva, we say the RF for the unit j is 2 dva and its eccentricity is 7 dva.

We study characteristics of receptive fields of units in deep convolutional networks. The receptive field size is a crucial issue in many visual tasks, as the output must respond to large enough areas in the image to capture information about large objects. We introduce the notion of an effective receptive field size, and show that it both has a Gaussian distribution and only occupies a fraction of the full theoretical receptive field size. We analyze the effective receptive field in several architecture designs, and the effect of sub-sampling, skip connections, dropout and nonlinear activations on it. This leads to suggestions for ways to address its tendency to be too small.

Learning with recurrent neural networks (RNNs) on long sequences is a notoriously difficult task.

For an example unit j, if it shows activity for input images having white pixels at 6 dva to 8 dva, we say the RF for the unit j is 2 dva and its eccentricity is 7 dva.